DOCUMENT RESUME 



ED 081 548 



56 



RC 007 257 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 



PUB DATE 

GRANT 

NOTE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Payne, T. V. ; Medina, Edward 

Educational Accomplishment Audit Report, FY 1971-72- 
[Southeastern New Mexico Bilingual Education 
Program. ] 

Artesia Public Schools, N. Mex. 
Bureau of Elementary and Secondary Education 
(DHEW/OE) , Washington, D. €• Div. of Bilingual 
Education* 
72 

OEG-0- 9-410028-3443 (280) 
19p. 

MF-$0.65 HC-$3. 29 

♦Bilingual Education; ♦Comparative Analysis; 
Educational Improvement; ♦Evaluation Criteria; 
Minority Group Children; Program Effectiveness; 
♦Program Evaluation; ♦Spanish Speaking 
♦Southeastern New Mexico Bilingual Program 



ABSTRACT 

The audit report (1971-72) on the Southeastern New 
Mexico Bilingual Education Program, under the sponsorship of the 
Artesia Public Schools, critiques the project evaluation with 
notations on comparative findings of the evaluation and audit* The 
report is divided into 4 sections: (1) introductory and general 
comments on project evaluation quality and the comparative findings 
of the project evaluation and the audit; (2) detailed critique of the 
comprehensive evaluation based on instruments used, data collection, 
data analysis, and data analysis presentation; (3) description of the 
auditor's on-site findings and their correlation with the evaluator's 
data and reports on a component-by-component basis, and a summary of 
consistencies and discrepancies; (4) recommendations for evaluation 
design revisions with a rationale for each; and (5) confirmation or 
questioning of the need for program modifications proposed in the 
project evaluation. (Author/NQ) 
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EDUCATIONAL ACCOMPLISHMENT AUDIT REPORT 

py mi- 7:2, 

This audit report on the Southeastern New Mexico 
Bilingual Education Program (0EG-0-9-*H0028-3H3-(280)) is 
made in compliance with the contractual agreement, legal pre- 
scriptions, and official directives under the provisions of 
Title VII of Public Law 89~10, as amended, for the establish- 
ment of bilingual education programs. The program is funded 
through. the Artesia (New Mexico) School District with Bill 
Gutierrez as Program Director. 

The audit report as submitted herein, is a critique 
concerning the project evaluation with notations on compar- 
ative findings of the project evaluation and the audit as 
re-.l ated to the -performance and process objectives identified 
in the evaluation design instrument. The auditor, in this 
report, will confirm or question the need for program modi- 
fications which have been proposed as a result of the project 
evaluation and onsite observations and visitations. 

Scope of the Audit Report 

The audit report follows rather closely the Suggested 
Audit Report Content Areas received from the United States 
Office of Education and made a part of the contractual agree- 
ment between the auditor and the program director, as follows 

1. Introductory and general comments concerning the 
quality of the project evaluation and the comparative 
findings of the project evaluation and the audit, 

2. Detailed critique of the comprehensive evaluation 
conducted, by component, based on an assessment of the 
instruments used, data collection, data analysis, and 

the manner in which data analysis is presented. 

3- Description' of the audit contractor 1 s on-site 
visit findings and their correlation with the Project 
Evaluator's data and reports on. a component by component 
basis; summary of consistencies and discrepancies. 

4. Recommendations for revisions in the evaluation 
design, including a rationale for each recommendation. 
Since the auditor's objectivity can be retained only if 
the selection of a specific corrective action is a local 
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decision, the audit contractor wiM provide general 
rather than specific recommend ations, posing several 
alternative actions or possible sources of assistance 
to the Project in correcting the deficiency. 

5. Confirmation or questioning of the need for pro- 
gram modifications which have been proposed as a result 
of Project evaluation. 

Each of the audit areas described above will be re- 
ferred to as sections with section numbers agreeing with the 
audit description number. 

SECTION I - INTRODUCTORY AND GENERAL COMMENTS 

The Southeastern New Mexico Bilingual Program, 
OEG-O-9-4 1 0028- 3^5- (280) , is under the sponsorship of the 
Artesia (New Mexico) Public Schools through the Educational 
Services Center. The Project is under the leadership and 
supervision of the Project Director and other available 
specialists from within the school system. 

Six public school districts (Artesia, Carlsbad, 
Roswell, Lake Arthur, Hagerman, and Dexter) with supplemen- 
tary help from Eastern New Mexico University- Ro swell Campus 
and the New Mexico State Department of Education have cooper 
ated to implement the Bilingual Program in the Lower Pecos 
Valley of New Mexico. The cooperating schools are partici- 
pating in the planned program on a daily basis for a period 
of nine(9) months. Artesi 3 is in the third year develop- 
ment phase while subsequent diversification is in the second 
year development phase in the remaining communities. 

Each classroom component operates within the regular 
hours of the respective elementary school. All classes 
participating in the project are housed in regular cl ass- 
rooms within the confines of an elementary school plant. In 
all components where monolingual laboratory teacher is i n - 
volved, bilingual sessions are provided within the morning 
schedule because the teacher aide assigned to each class- 
room being serviced works in the classroom only in the 
mo rn i ng . 

The effectiveness of an evaluation is primarily 
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determined by how well the evaluating techniques quantify 
through objective measurements the quality and extent of 
pupil learning for which the project is chiefly responsible. 
After careful examination of the evaluation design, techniques 
employed in the evaluation, procedures foil owed, and purusual 
of the written evaluation report it can be concluded that the 
major objective has been to measure learning out comes and the 
progress in learning. The evaluation, although concerned 
with inputs, has concentrated its evaluative judgments on 
outputs — progress toward achievement of stated project 
objectives. 

Standardized instruments, uniformly applied to ex- 
perimental (project participants) groups and control groups 
have been utilized for objective measurements. Baseline 
references and data have been rollowed in reporting group 
accomplishment. The instruments used were selected because 
of their generally high acceptance as standardized measuring 
devices and high validity in normal applications. Analysis 
of test results have been divided into four groups: 

1. Dominant Spanish speaking children in the bilingual 
componen t s . 

2. Non-Dominant Spanish speaking children in the 
bilingual componen:. 

3. Dominant Spanish speaking children in the lab- 
oratory component. 

A. Non-Dominant Spanish speaking children in the 
laboratory component. 

Informal assessment based on observations, teacher 
made measuring instruments, and student performance were 
utilized to provide feedback du:ing the teach i ng- 1 earn i ng 
process as instruments (or processes) for Project evaluation 
purposes. Criterion referenced tests for evaluation purposes 
received only minimal use. 

The evaluation design, procedures, and techniques 
have emerged as the program has developed. Conferences be- 
tween the Project Director, the Evaluator, and the Auditors 
throughout the year have been effective in (a) creating 
channels of communication for better understandings and (b) 
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elimination of irrelevant procedures and the refinement of 
instruments to better assess the level, or degree, of a ccom- 
plishment of program objectives. 

The utilization of the Stanford Achievement Test at 
the first grade level does not provide a pre-test baseline 
since it is not applicable for use with children just be- 
ginning school. Comparisons of first grade progress has been 
in relation to a 0.0 assumed baseline for both Project and 
Control children at the first grade level. 

In general, the evaluation was well done and has 
considerable merit. The evaluation results point to some 
needed alterations in the program and provide evidence of 
successful pupil growth in a variety of areas. Analysis of 
the evaluation report as it relates to successful accomplish- 
ment and areas of concern will be dealt with in subsequent 
sections of this report. 

Commen ts and clarifications made herein and in sub- 
sequent recommendations are not intended to lead to any 
hasty conclusions with regard to the adequacy of the Evalu- 
ation Design and Evaluation Procedures as submitted. The 
evaluation instruments have proven to be generally satis- 
factory, and the evaluation report at this time is quite 
adequate. This is especially true when one considers that 
this is an evaluation of a developing program with few 
guidelines or precedents upon which to base data interpre- 
tation and conclusions. Also, it must be recognized that 
any attempt to evaluate a b i 1 i ngu a 1 - b i cu 1 tu r a 1 program with 
instruments which have been standardized on the broader 
population in which the English speaking population repre- 
sents a strong majority presents some problems that have not 
yet been adequately resolved. 

SECTION II - CRITIQUE OF EVALUATION REPORT 

The Evaluation Design . The Evaluation Design was 
developed to measure the extent to which performance objec- 
tives were attained. The design is well organized and meets 
the basic needs for effective program evaluation. 
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The Evaluation Ins truments. Some difficulty has been 
experienced in test instruments. As clarified in the Evalu- 
ation Report, several of the test instruments are less than 
totally satisfactory for accomplishing effective measu remen t s 
of growth toward the achievement of established objectives. 

It would seem advisable for the project to consider 
criterion referenced tests or seek new or additional tests 
that offer greater utility and validity in testing bilingual 
child ren . 

Control Groups . Care taken in selection of control 
groups has assured randomly matched counterparts. Some dif- 
ficulties occure as "control 11 families move and tend to de- 
plete the number needed for meaningful comparisons. 

Audit of Performance 

Objectives Accomplishment 

Ob j ec t i ve : Dominant Spanish speaking children in the 
Bilingual Program will learn 5% more of the standard curric- 
ulum as compared to dominant Spanish speaking children of 
comparable age, grade and cultural groups who did not partic- 
ipate in the p rog ram . 

In comparing first grade bilingual pupil gain with 
control pupil gain Stanford Achievement Test results show 
control group to have slightly greater gain in (l) word, 
paragraph, and vocabulary and (2) word study skills. There 
was no difference between groups in (3) arithmetic. It 
follows that the above objective was not accomplished. 

The second grade bilingual group, when compared with 
the control group showed significant gains in all areas 
tested. The objective of 5 percent gain was reached and 
extended to 8.36 percent gain. 

The third grade bilingual group, when compared with 
the control group, showed significant gains in all test 
areas. The 5 percent gain objective was extended to 1 A . 30 
percent . 

Conclusions: While the first grade groups were very 
close with the control group having a slight edge, the gains 



of second and third grade bilingual groups was well beyond 
the 5 percent objective. 

Objective : Non-Dominant Spanish speaking children 
in the Bilingual Program will learn 5 percent mo re of the 
standard curriculum as compared to Non-Dominant Spanish 
Speaking children of comparable grade, age, and cultural 
group who have not participated in the program. 

The first grade bilingual and control groups each 
had only one non-dominant Spanish speaking child. The 
number is too small for any meaningful analysis. 

The second grade bilingual group showed significant 
gain (16. 0k%) over the control group in Words, Paragraph, 
and Vocabulary section of the S . A. T . and 7.84% gain over the 
control group in Spelling and Word Study Skills. 

The third grade bilingual group scored lower than the 
control group on both sections of the test. 

Conclusion: Although there is some evidence ir- favor 
of the bilingual group at the second grade and evidence 
favoring the control group at the third grade level the small 
number of students involved make it impossible to conclude 
with any degree of certainty regarding the accomplishment 
of the ob j ect i ve . 

Ob j ec t i ve : Non-Dominant Spanish speaking children 
in Lab Group will score as high as, or higher than, the Non- 
Dominant Spanish speaking children of comparable grade, age, 
and cultural group who, have not participated in the Program. 

The laboratory group showed sliy.t loss over the 
control group in all phases of the test* Howe ve r , this is 
not a realistic comparison because only one child was in this 
classification of the Program students. 

Conclusion: The number sampled is not adequate to 
arrive at any conclusion regarding the accomplishment of the 
objective* 

Objective: Dominant Spanish speaking children in the 
Lab Group will learn 5 percent more of the standard curric- 
ulum as compared to Dominant Spanish speaking children of 
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comparable age, grade, and cultural g roups who have not 
participated in the Program. 

The control group was slightly ahead of the labora- 
tory g roup by .27%. 

Conclusion: The Program group failed to meet its 
5% greater growth objective. 

Significant Observation: It is worthy of notation 
at this point that the test measured I.Q. scores ranged 
several points difference between the Program students ard 
their counterparts in rooms not involved in the program. 
This difference consistently favored the control groups. 
Since this I.Q. difference was greater than 5% , and when 
taken into account, it is only reasonable to conclude that 
children in the Program did relatively better than test re- 
sults s how * 

Objective: All students in the Program will score 
higher individually on the Spanish version of the Peabody 
Picture Vocabulary Test than comparable students (age, 
grade, and cultural background) not participating in the 
Program by the following standards: (a) Dominant Spanish 
speaking students by 5%, (b) Non-Domiant Spanish speaking 
by ]0%. All students in the Program will collectively score 
higher on the Spanish version of the Peabody Picture Vocabu- 
1 a ry Test than compa rable students (age, grade, and back- 
ground) not participating in the Program by the following 
standards: (a) Dominant Spanish speaking students by ]0% 
and (b) Non-Dominant Spanish speaking students by ]2%. 

The test scores indicate that Project students at 
all three grade levels surpassed those in the control group 
by considerable margins. There we^e some individual excep- 
tions. 

Conclusion: Students in the Program did score 
higher than the control group by scores sufficiently high 
to conclude that the objective was successfully achieved. 

q Object i ve : Individual comparative scores on the 

ERJ.C 
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English version of the Peabody Picture Vocabulary Test be- 
tween students in the Program and comparable control students 
will show the following: (a) Dominant Spanish speakiry Pro- 
gram students will excell by $% and (b) Non-Dominant Spanish 
speaking students will score as high as control oroup stu- 
dents. Collective comparative scores on the English version 
for the Peabody Picture Vocabulary Test bet we en students in 
the Pron-am and comparable control students will show the 
following: (a) Dominant Spanish speaking Program students 
will excell by 10% and (b) Non-Dominant Spanish speaking 
Program students will score as high as control group students. 

In individual comparative scores expected goals were 
reached by kh% of first grade Program students, by 50% of 
second graders, and by only 7% of third graders. In compar- 
ing collective group scores the bilingual dominant Spanish 
speaking first graders exceeded the control group by 1 k% , and 
non-dominant Spanish speaking exceeded the -.ontrol group by 
16%. At the second grade level Program stuJtnts excelled by 
5% and 2 7% . Third grade Program students excelled by 1 % and 
5%. 

Conclusion: Although the gains by ■ .ie dominant 
Spanish speaking Program children in second and third grades 
failed to meet the objective, all first graders and the non- 
dominant Spanish speaking Program children met or exceeded 
the ob j ec t i ve . 

Objective: All students in the Program will have a 
5% better self-image than control students of comparable 
age, grade, and cultural group not in the Program as measured 
by a self-image scale. 

All Program groups showed positive gains on the self- 
image test while all control groups s howed a loss in self- 
image ranging from a loss of 5-3$ to 17.1%. 

Conclusion: The first and second grade groups showed 
9.8% and 6.6% gain respectively while the control groups 
showed 5.3% find 17*1i losses. The third grade Program group 
showed only .07% gain while its counterpart showed an 11% 

IC 

loss. It must be concluded that the objective was reached 
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and exceeded. 

General Audit Observations 

From records in the Project Director's office and at 

other locations throughout the Program locations the auditor 

was able to ascertain the following: 

1. A we 1 1 planned, scheduled, and con due t e*v se r i es 
of field trips made it possible for the students to visit 
a number of worthwhile areas and activities in the com- 
munity a rea . 

2: Five inservice workshops on a variety of relevant 
topics were held. These were generally all-day sessions 
and were exceptionally well attended. The auditor had 
occasion to observe participants at two of these inse r vice 
sessions and must conclude that participant interest was 
except i onally high. 

3. Mr. Albino Baca, formerly a teacher in the bi- 
lingual Program, was moved to the assignment of Curric- 
ulum Specialist for the Program, One of his major 
respons i b 1 i I i tes was to supervise the production of 
instructional materials. The Program has developed and 
printed a number of books and other materials. These in- 
clude a remembrance story collection, adventrue stories, 
stories by students, and an ecological book. Several 
books have workbooks and teacher manuals. A curriculum 
guide for bilingual education, although not yet completed, 
shows consideralbe progress and much potential. It 
should be helpful to those involved in the Program and 
may well serve as a model for other schools to use. 

4. One of the strong features of the project has 
been the close home-school -commun i ty coordination. This 
has come about because of the interest of all participants 
in the project with special emphasis and effort provided 
by the Home Coordinator, The Home Coordinator has main- 
tained a comprehensive daily log of activities. Mrs. 
Anna Almanza has worked most actively and aggressively 

in relating the project to the home and involving the 
parents in school activities. She has assisted teachers 
and aides in furthering school-home relationships. She 
has worked effectively with individuals, organizations, 
and town groups. A great deal of counselling with parents 
regarding their children and school progress has helped 
further the home 1 s appreciation for, and interest in, 
the bilingual Program. 

5. The college program for paraprof ess iona I s has 
operated in a successful manner. Following is a summary 
of the training component of the Program. College courses 
offered through ENMU-Roswell Branch were conducted 
regularly at Lomita School in Artesia. Courses offered 
were in keeping with the aides 1 degree plans. Analysis 

of the final grades for the Fall Semester and Mid-term 
grades for the Spring Semester indicates the aides were 
satisfactorily applying themselves and the collective 
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grade-point each semester wo s welt above a 2 . on the k 
point system. Fall final grades: A-23, B - 3 3 » C-26, and 
F-8. Spring mid-term grades: A- 1 2 , B-k7 , C - 3 7 , D-7 , 
and F-1. Staff members were well qualified, experienced 
teachers. Degree Plans for all participants have been 
properly prepared. 

6. The Advisory Council held six meetings during 
the year with attendance ranging from 10 to 26 members. 
Meetings, for the most part, were of dual purpose: (1) 
to keep the committee informed regarding the progress of 
the Program and (2) to provide inservice-type training 
regarding bilingual education. The agendas were well 
planned and minutes were properly written following each 
mee t i n g . 

7 . A suggested curriculum guideline has be en devel- 
oped. It contains recomme nded daily time allotments. The 
course con tent and suggested strategies has been included 
for: B i -cu 1 tura 1 i sm in the Bi-lingual School, Spanish 
Materials, Spanish Concepts and Objectives, Bilingual 
Mathematics, English as a Second Language, Language Arts 
in the Bilingual Program, Language Arts Concepts and 
Objectives, Ciencias 1971-72, Seguridad Bilingual 1971- 
72, and Bilingual Health. 

8. Purchase orders showed that $85^3 was allocated 
to the Bilingual Program from the state school enrichment 
fund. The systematic record keeping system maintained 
separate records for purchases made from this special 
account. An equipment inventory is maintained. The in- 
ven to ry shows the quantity, location, value, and person 
responsible for each inventoried item. Where appropriate 
serial numbers and/or other identification is included. 
The latest inventory is dated January, 1972. A new in- 
ventory was to be taken at the close of the current school 
year. Purchase orders for supplies, materials, equipment, 
and books are maintained in the office of the project 

d i rec tor . 

SECTION III - AUDITORS' ON-SITE. VISITATION 

During the course of the 197 1 -72 fiscal year '.he 
auditing team spent a total of 1^ man-doys in on-site visita- 
tion. Interim reports we re submitted ear] i er . The following 
is a report of visitations near th^ end of the teaching year. 

The specific purposes of the on-site visitation were 
(1) to assess the role and input of Artesia Public School 
Administration in the Title VII Program, (2) to determine 
whether the teacher aide training program objectives we re 
being accomplished, (3) to observe whether teachers were 
making better use of their aides, a weakness identified 
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during earlier visits, and (k) to study raw data and other 
pertinent information related to the evaluation of the Program. 

Administration of the Program . The Project Director 
and the Curriculum Specialist have f o rmed a very effective 
team of bilingual educators which has given a realistic and 
educationally sound meaning, unders tood by all participants, 
to the philosophy of bilingual education and to the content of 
of its curriculum. 

On-site visits to the various centers located in 
Artesia, Carlsbad, Ro swell and Lake Arthur have served to 
establish that the respective principals, with perhaps one 
exception, are most cooperative and know the WHY of the Pro* 
gram* The Project Director and Curriculum Specialist are 
consult ed by the Principals in matters relating to personnel, 
parent involvement, ano pupil participation involved in ^11 
facets of the project. It was found that most teacher i • I 
teacher-aides were very enthusiastic about the entire SiPngual 
activity and well-informed as to the HOW of implement ng me 
curriculum. The school community has come to endorse, accept 
and understand the bilingual program as is currently being 
ad ministered by the Artesia Public Schools. 

The Southeastern New Mexico Bilingual Program per- 
sonnel continue to express a willingness to work in modifying 
and adapting the curriculum content to meet the individual 
educat iona 1 needs of children attending the various elementary 
schools within the latter 1 s broad goals and concepts. Schoo 1 
Administrators (Superintendents, Principals, Curriculum 
Coordinators, etc.) should continue to work toward developing 
viable working relationships with all those assigned to work 
in the bilingual program so as to enhance an effective edu- 
cational experience for the children involved and to irradi- 
cate the myth that Spanish surname children en ro lied in these 
early graues are identical robots rather than children which 
are Indeed different human beings, with emot Ions and feelings, 
deserving of an opportunity to succeed. It is hoped, there- 
fore , that all staff members not presently Involved in the 
bilingual program accep^lt as an integral part of the total 
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school goals and concepts rather than as something "special" 
and without continuity. 

Recuritment and assignment of pupils as participants 
in the bilingual program should involve more concerned people. 
It is difficult to determine and interpret the criteria used 
for recruitment and assignment of pupils ot some centers 
when vie we d in light of children in the program. 

Teacher-Aide Training Program . The Grant Package 
Proposal for FY 1971-72 included an ambitious, but realistic, 
program for the pa r a- p r of es s i o n a I training component of the 
project. Although the College of Artesia had been the main 
Center for college training of para-professional project 
participants, its closing caused a temporary halt to the 
course work plans. However, an agreement was reached with 
Eastern New Mexico University - Rosv, e 1 I Campus to continue the 
training of the pa r a - p r of es s i on a 1 s . At the present time 
ENMU-Roswell Campus does offer to para-professionals a pro- 
gram of studies design which will enable the participants to 
eventually acquire a Bachelor of Arts Degree in elementary 
education with a major in the teaching of ethnic groups and 
a minor in Spanish. Some thirty participants are taking ad- 
vantage of this opportunity. Although the latter will be 
explained in more detail in the proposal for continuation of 
the Program for FY 1972-73 it is cited in this report, not 
because it is another program for another degree, but be- 
cause it is an example of how innovative programs can be 
developed when the efforts of colleges and public schools 
work closely together. Cultural Anthropology, Ethnology of 
the Southwest, Selected Topics in Spanish (a beginning field 
course in Bilingual Instruction), Hispanic Culture and Civi- 
lization, and all levels of Spanish courses are but a few of 
the prescribed offerings. It is the feeling of the auditing 
team that the pre-service and in-service training programs 
as implemented have ^-"ovided the participants with a vast and 
useful bcdy of knowledge regarding modern theories of human 
growth and development and a realistic philosophy of edu- 
cation. We agree with the Project Director when he states 
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that M the teacher-aide training component of the project has 
offered stability and scope toward the gradual training of 
Bilingual teachers to fill the vast needs of Southeastern New 
Mexico." Another significant strength of the prescribed 
training component (preservice and in-service) is that 
although methods for implementation of the curriculum are a 
basic part of the agenda, it is child-oriented when one 
studies the content and guidelines offered to the partici- 
pants. Parents , teachers, teacher-aides and pupils are in- 
volved in the planning. The input provided in this manner 
is valued by both the Project Director and the Curriculum 
Specialist. Incidentally, the latter two professionals go 
about their business with dedication and not with assumed 
superiority as two "general es" of a militaristic extravaganza. 
Their sincere dedication serves as a model for other partici- 
pants in the program. 

Teacher Use of Teacher-Aides. This component of the 
program appears to need more emphasis during FY 1972-73- It 
does not mean, how ever, that it has been identified as a 
major weakness of the program. It is a major component of 
any effective bilingual-bicultural program. If the teacher- 
aide is to function as a second voice for a monolinual 
teacher by assisting with the Spanish instructional reinforce- 
ment, the implications of importance are obvious. 

At several of the centers visited teachers were 
hesitant to allow the teacher-aide to assume the instruction 
of the related curriculum in Spanish but chose to have her 
performing English-oriented activities as "correcting," 
♦'checking," "spelling," and building "exhibits." It should 
he stressed to all participating teachers that their aides 
are part of the teaching team and that planning for the day's 
activities is indeed a cooperative enterprise involving both. 
Significant improvements were noticed in the foil ow -up visit. 
However, more needs to be done if teacher aides are to be- 
come really useful tools for instruction in a bilingual aura 
rather than as a crutch in an antiseptic teaching exercise. 
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The Evaluat ion. Although numerous problems arose 
during the evaluation process cooperative efforts and sincere 
interest in the program made it possible to meet all obstacles 
with satisfactory alternative solutions. The evaluation 
report as submitted has been thoroughly critiqued elsewhere. 
Suffice it here to merely comment that the evaluation was 
done in a very professional manner and presented with as much 
objectivity as was possible. There was no evidence of any 
attempt to whitewash any phase of the program. On several 
occasions explanations were made regarding some testing re- 
sults. These appear to be logical explanations and such 
conclusions were arrived at only after very thoughtful con- 
sideration. 

Cone 1 us ion . The auditing experience, particularly the 
on-site visitation, has been a most pleasant one. The cour- 
tesies extended by all of the Bilingual Program staff and 
school administrators are indeed appreciated and most grati- 
fying. The auditing team sincerely submits that the South- 
eastern Bilingual Program might well serve as a model for 
this region, as for three years it has dared to be creative, 
innovative, flexible, and unafraid of publishing the factual 
results of its effort using as a base its belief that "this 
program is a vehicle and pedigogical tool to better prepare 
children to function as successful citizens in a democratic 
society." It is firmly believed that this program is bring- 
ing to Artesia and environs interracial unity and under- 
standing. 

SECTION IV - REVISIONS IN EVALUATION DESIGN 

Previous Evaluation Designs have been designed to be 
as compatible with the Artesia Public Schools testing and 
evaluation program as possible. It has been felt that if 
the bilingual program is to become a regular and vital com- 
ponent of the school curriculum it should incorporate as 
much of the regular program as feasible. The evaluation 
O ;sign reflects this thinking. 

ERIC 
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Basically the Evaluation Design has served its fun- 
ction well. It has attempted, and with measurable success, 
to establish levels of growth in several areas. The S tanf ord 
Achievement Tests and the Peabody Picture Vocabulary Test 
have measured cognitive growth and provided information to 
compare students with the Artesia school children in general. 
Self-image scales and teacher judgmer^s >elp to measure 
important areas in the affective domain. 

The Evaluation Design has been constructed in an 
organized manner that makes it easy to identify the Projects 
objectives and the tools to be used for measuring accomplish- 
ments in these areas. It is recommended that the structural 
pattern of the Evaluation Design for 1972-73 remain essentially 
the same. 

It may be desirable to consider some changes in kinds 
of measuring tools and procedures. It would appear that the 
bilingual ana laboratory components have process goals and 
purposes so varied from the usual school program that special 
measuring techniques become necessary. 

The formal evaluation has given only small consider- 
ation to the te-cher's analysis of individual pupil growth. 
It would appear th t this is an important source of evalu- 
ative information that has been largely negi -cted. The 
auditing team would recommend greater utilization of teacher 
assessment of individual students. 

Criterion referenced examination (tists) , developed 
at th<* local project level would f < ■ a clcser attention to 
the stated objectives. Locally d^ve;oped tests would meet 
two common basic concer ^ regarding the evaluation process: 
(l) English oriented and sifncrdized tests reflect a bias 
that is disadvantageous to the dominant Spanish speaking 
child and (2) Available te><.s do not adequately measure all 
of the planned growth changes deemed important in the bi- 
lingual education program. 

Teacher developed me ring devices would be another 
lfvel of the materials develooment component of the project. 
Thase might be directly relied to the special materials 
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being developed by the project as well as other materials and 
experiences orovided through the project. 

The ; uggestions made here are not to be interpreted 
as fa vo ring elimination of present testing program activities. 
It is possible that some parts of the present program do 
measure objectives accomplishment better than other instru- 
ments. In this case they should be retained. Criterion 
based and teacher developed measuring instruments would then 
be used to supplement the present evaluation process. 

SECTION V— NEEDS FOR PROGRAM MODIFICATION 

The Southeastern New Mexico Bilingual Program has 
just completed its third successful year of operation. The 
program in the Artesia Public Schools has advanced through 
the third year (grade level) of school. 

The successes and weaknesses of the project have been 
identified through visitations, ob s e r va t i on s, informal 
assessments, and the formal evaluation of the project. 

On the basis of these opportunities for assessing 
the program the auditors present the following suggestions: 

1. Establish more clearly the differentiation between 
the bilingual class rooms and the laboratory class rooms i n 
order that the strengths and weaknesses of each might be more 
clearly identified* This is also suggested as a means of 
hel p i ng school personnel not identified with the program and 
interested townspeople better understand the program. 

2 . % Establish some framework for analysis of cos t - q ua 1 i ty 
relationships of both the bilingual and the laboratory com- 
ponents. These should be developed in such manner that the 
dollar value of services received can be compared with the 
pupil cost in the "regular 11 classroom. 

3 . Some systematic assess me nt should be made to compare 
the 11 depart me n t e> 1 i zed " bilingual classrooms with those that 
are self-contained. 

4. Earlier assessments have shown that students 1 self- 




image suffers significantly at the intermediate grade levels. 
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Serious consideration for extending the bilingual program 
into the fourth year (grade) with special emphasis being 
placed on the affective behavioral domain should be a priority 
item for the Artesia Public Schools. 

5 . Elementary building principals need to become more 
actively involved. Ultimately the program will become their 
responsibility as part of the elementary school program. 
Every effort should be made to involve them in all inservice 
activities. Likewise, principals should actively engage in 
future project planning. 

6. The Project Director might consider the possibility 
of negotiating with one of New Mexico's colleges to offer 

an extension course/s in conversational Spanish. This would 
be primarily for teachers in the project but open to others 
if they can be accomodated. College credit should be avail- 
able. 

7. Efforts should continue to encourage teachers to plan 
for, and utilize, teacher aide services more extensively as 
bilingual adjuncts in the classroom. 

8. Time utilization is important in order that both 
teacher and aide function at peak efficiency. Careful 
planning should become a part of role expectations for ail 
personnel in the program In order that the best possible 
utilization of available time and professional talents will 
be assured. 

CONCLUS IONS 

The Southeastern New Mexico Bilingual Program has 
successfully completed its third year of operation. During 
this time the program has expanded upward and into other 
cooperating school districts. 

Although staffing primary grades with well qualified 
teachers is becoming increasingly difficult, the Southeastern 
New Mexico Bilingual Program has successfully met this 
obstacle and the program has moved forward with no significant 
interruptions. 
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Mr. Bill Gutierrez, Project Director, has worked 
unceasingly toward the accomplishment oT project goals. He 
has exerted positive leadership in involving staff members in 
planning and program development. Mr. Gutierrez enjoys high 
respect from his staff, the administration, and the downtown 
commun i ty . 

The staff has proven itself to be dedicated and 
anxious to develop the best possible bilingual program to 
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measure as strongly in the evalution process as aimed for 
in the objective. Yet, generally speaking the project has 
demonstrated success. Although no apology is needed, ex- 
planation is presented regarding the relatively low measured 
I.Q.'s of project students when compared to other comparable 
groups . 

The project auditors have been well received by 
project staff members and administrators of al! schools 
cooperating in the project. Records, reports, and materials 
were always available when needed. 

The Southeastern New Mexico Bilingual Project and 
its staff are to be commended for providing a forward thrust 
program which is contributing to the educational breakthrough 
for effective education for the dominant Spanish speaking 
child in the southwest. 

Respectfully submitted: 

!. V, Payne, Chief Auditor 

Edward Medina, Auditor 
June 39, 1972 




